Distance-velocity disambiguation in hybrid light detection and ranging devices

ABSTRACT

The subject matter of this specification can be implemented in, among other things, a system that includes a first light source to produce a pulsed beam and a second light source to produce a continuous beam, a modulator to impart a modulation to the second beam, and an optical interface subsystem to transmit the pulsed beam and the continuous beam to an outside environment and to detect a plurality of signals reflected from the outside environment. The system further includes one or more circuits configured to identify associations of various reflected pulsed signals, used to detect distance to various objects in the environment, with correct reflected continuous signals, used to detect velocities of the objects. The one or more circuits identify the associations based on the modulation of the detected continuous signals.

TECHNICAL FIELD

The instant specification generally relates to distance and velocitysensing in applications that involve determining locations andvelocities of moving objects. More specifically, the instantspecification relates to hybrid lidars, in which distance and velocityare measured using separate channels.

BACKGROUND

Various automotive, aeronautical, marine, atmospheric, industrial, andother applications that involve tracking locations and motion of objectsbenefit from optical and radar detection technology. A rangefinder(radar or optical) device operates by emitting a series of signals thattravel to an object and then detecting signals reflected back from theobject. By determining a time delay between a signal emission and anarrival of the reflected signal, the rangefinder can determine adistance to the object. Additionally, the rangefinder can determine thevelocity (the speed and the direction) of the object's motion byemitting two or more signals in a quick succession and detecting achanging position of the object with each additional signal. Coherentrangefinders, which utilize the Doppler effect, can determine alongitudinal (radial) component of the object's velocity by detecting achange in the frequency of the arrived wave from the frequency of theemitted signal. When the object is moving away from (towards) therangefinder, the frequency of the arrived signal is lower (higher) thanthe frequency of the emitted signal, and the change in the frequency isproportional to the radial component of the object's velocity.Autonomous (self-driving) vehicles operate by sensing an outsideenvironment with various electromagnetic (e.g., radio, optical,infrared) sensors and charting a driving path through the environmentbased on the sensed data. Additionally, the driving path can bedetermined based on positioning (e.g., Global Positioning System (GPS))and road map data. While the positioning and the road map data canprovide information about static aspects of the environment (buildings,street layouts, etc.), dynamic information (such as information aboutother vehicles, pedestrians, cyclists, etc.) is obtained fromcontemporaneous electromagnetic sensing data. Precision and safety ofthe driving path and of the speed regime selected by the autonomousvehicle depend on the quality of the sensing data and on the ability ofautonomous driving computing systems to process the sensing data and toprovide appropriate instructions to the vehicle controls and thedrivetrain.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of examples, and not by wayof limitation, and can be more fully understood with references to thefollowing detailed description when considered in connection with thefigures, in which:

FIG. 1 is a schematic illustration of advantages of a hybrid lidarsensor that uses velocity-distance disambiguation, in accordance withsome implementations of the present disclosure.

FIG. 2 is a diagram illustrating components of an example autonomousvehicle that uses one or more hybrid lidars with distance-velocitydisambiguation, in accordance with some implementations of the presentdisclosure.

FIG. 3 is a block diagram illustrating an example implementation of ahybrid lidar that uses distance-velocity disambiguation, in accordancewith some implementations of the present disclosure.

FIG. 4A, FIG. 4B, and FIG. 4C illustrate example implementations offrequency encodings that can be used to modulate a continuous beamoutput by a hybrid lidar, in accordance with some implementations of thepresent disclosure. FIG. 4A illustrates a symmetric “staircase”frequency modulation that uses eight different frequency values, eachimplemented for a particular time interval. FIG. 4B illustrates anasymmetric staircase frequency modulation that uses fifteen differentfrequency values each implemented for a specific time interval. FIG. 4Cillustrates a directional staircase with resets, in which frequency ismonotonically increased from a minimum frequency to a maximum frequencyfollowed by a reset back to the minimum frequency.

FIG. 5 illustrates identification of a range of distances by a hybridlidar system that uses an example frequency encoding of a continuousbeam, in accordance with some implementations of the present disclosure.

FIG. 6A and FIG. 6B further illustrate identification of a range ofdistances by a hybrid lidar system using an example directionalfrequency staircase encoding, in accordance with some implementations ofthe present disclosure. FIG. 6A illustrates the frequency encoding ofFIG. 4C, which uses five different values of frequency. FIG. 6Billustrates identification of the ranges of distances based on beatingfrequencies by a hybrid lidar that uses directional frequency staircaseencoding of FIG. 6A.

FIG. 7 depicts a flow diagram of an example method of distance-velocitydisambiguation in hybrid lidars, in accordance with some implementationsof the present disclosure.

FIG. 8 depicts a flow diagram of an example method of associating rangesof distances to reflecting objects using returns generated by acontinuous beam with imparted angle modulation, in accordance with someimplementations of the present disclosure.

SUMMARY

In one implementation, disclosed is a system that includes a first lightsource configured to produce a first beam comprising one or more pulses,and a second light source configured to produce a second beam, whereinthe second beam is a continuous beam. The system further includes amodulator configured to impart a modulation to the second beam and anoptical interface subsystem configured to: transmit the first beam andthe second beam to an outside environment, and receive, from the outsideenvironment, a plurality of received (RX) signals caused by at least oneof the first beam or the second beam. The system further includes one ormore circuits configured to determine that a first RX signal of theplurality of RX signals and a second RX signal of the plurality of RXsignals are reflected by a same object, wherein the first RX signal isrepresentative of a distance to the object and the second RX signal isrepresentative of (i) a velocity of the object, and (ii) an interval ofpossible distances to the object, the interval of possible distancesbeing identified based on the modulation of the second RX signal.

In another implementation, disclosed is a sensing system of anautonomous vehicle (AV), the sensing system including an optical systemconfigured to: produce a first beam comprising one or more pulses, thefirst beam centered at a first frequency, produce a second beam, whereinthe second beam is a continuous beam centered at a second frequencydifferent from the first frequency, impart an angle modulation to thesecond beam, transmit the first beam and the second beam to anenvironment of the AV, and receive, from the environment of the AV, aplurality of received (RX) signals caused by at least one of the firstbeam or the second beam. The sensing system further includes a signalprocessing system configured to: determine that a first RX signal of theplurality of RX signals and a second RX signal of the plurality of RXsignals are reflected by a same object in the environment of the AV,wherein the first RX signal is representative of a distance to theobject and the second RX signal is representative of (i) a velocity ofthe object, and (ii) an interval of possible distances to the object,the interval of possible distances being identified based on the anglemodulation of the second RX signal, and associate the distance to theobject, determined from the first RX signal, with the velocity of theobject, determined from the second RX signal.

In another implementation, disclosed is a method that includes producinga first beam comprising one or more pulses, producing a second beam,wherein the second beam is a continuous beam, imparting a modulation tothe second beam, transmitting the first beam and the second beam to anoutside environment, receiving, from the outside environment, aplurality of received (RX) signals caused by at least one of the firstbeam or the second beam; and determining that a first RX signal of theplurality of RX signals and a second RX signal of the plurality of RXsignals are reflected by a same object, wherein the first RX signal isrepresentative of a distance to the object and the second RX signal isrepresentative of (i) a velocity of the object, and (ii) an interval ofpossible distances to the object, the interval of possible distancesbeing identified based on the modulation of the second RX signal.

DETAILED DESCRIPTION

An autonomous vehicle can employ a light detection and ranging (lidar)technology to detect distances to various objects in the environmentand, sometimes, the velocities of such objects. A lidar emits one ormore laser signals (pulses) that travel to an object and then detectsarrived signals reflected from the object. By determining a time delaybetween the signal emission and the arrival of the retro-reflectedwaves, a time-of-flight (ToF) lidar can determine the distance to theobject. A typical lidar emits signals in multiple directions to obtain awide view of the outside environment. The outside environment can be anyenvironment in which the autonomous vehicle can operate, including anyurban (e.g., street) environment, rural environment, highwayenvironment, indoor (e.g., warehouse) environment, marine environment,and so on. The outside environment can include multiple stationaryobjects (roadways, buildings, bridges, road signs, shoreline, rocks,etc.), multiple movable objects (e.g., vehicles, bicyclists,pedestrians, animals, ships, boats, etc.), and/or any other objectslocated outside the AV. For example, a lidar device can scan an entire360-degree view and collect a series of consecutive frames identifiedwith timestamps. As a result, each sector in space is sensed in timeincrements Δτ, which are determined by the angular velocity of thelidar's scanning speed. “Frame” or “sensing frame,” as used herein, canrefer to an entire 360-degree view of the outside environment obtainedover a scan of the lidar or, alternatively, to any smaller sector, e.g.,a 1-degree, 5-degree, a 10-degree, or any other angle obtained over afraction of the scan cycle (revolution), or over a scan designed tocover a limited angle.

ToF lidars can also be used to determine velocities of objects in theoutside environment, e.g., by detecting two (or more) locations {rightarrow over (r)}(t₁), {right arrow over (r)}(t₂) of some reference pointof an object (e.g., the front end of a vehicle) and inferring thevelocity as the ratio,

$\overset{\rightarrow}{v} = {\frac{{\overset{\rightarrow}{r}\left( t_{2} \right)} - {\overset{\rightarrow}{r}\left( t_{1} \right)}}{t_{2} - t_{1}}.}$

By design, the measured velocity {right arrow over (v)} is not theinstantaneous velocity of the object but rather the velocity averagedover the time interval t₂−t₁, as the ToF technology does not allow oneto ascertain whether the object maintained the same velocity {rightarrow over (v)} during this time interval or experienced an accelerationor deceleration (with detection of acceleration/deceleration requiringadditional locations {right arrow over (r)}(t₃), {right arrow over(r)}(t₄) . . . of the object for t₃,t₄∈(t₁,t₂)).

Coherent lidars operate by detecting a change in the frequency of thereflected signal—the Doppler shift—indicative of the velocity of thereflecting surface. Measurements of the Doppler shift can be used todetermine, based on a single sensing frame, radial components (along theline of beam propagation) of the velocities of various reflecting pointsbelonging to one or more objects in the outside environment. A localcopy (referred to as a local oscillator (LO) herein) of the transmittedsignal can be maintained on the lidar and mixed with a signal reflectedfrom the target; a beating pattern between the two signals can beextracted and Fourier-analyzed to determine the Doppler shift andidentify the radial velocity of the target. A frequency-modulatedcontinuous-wave (FMCW) lidar can be used to determine the target'svelocity and distance to the lidar using a single beam. The FMCW lidaruses beams that are modulated (in frequency and/or phase) with radiofrequency (RF) signals prior to being transmitted to a target. RFmodulation can be sufficiently complex and detailed to allow detection,based on the relative shift (caused by the time-of-flight delays) of RFmodulation of the LO copy and RF modulation of the reflected beam.

FMCW lidars utilize high precision technology and are complex andexpensive devices. A less expensive option can be to use a hybrid lidardevice in which the ToF range-finding technology is combined with thevelocity-finding Doppler technology. A hybrid lidar can have two (ormore) lidar components. Each lidar component can output separate laserbeams and collect separate pieces of information about the target usingthe output beam. For example, a first lidar component can use pulsedbeams configured for accurate detection of a distance to the target. Asecond lidar component can output a continuous beam and detect Dopplerfrequency shift of the reflected signal for accurate detection of thetarget's velocity. The two lidar components can be combined on a singleplatform that allows for a concurrent transmission of the beams alongthe same optical path (while the hybrid lidar scans the outsideenvironment). When reflection from a single target object is detected,such a hybrid lidar can provide excellent functionality and determinethe distance to the object and the object's radial velocity with a highaccuracy. Yet on those occasions where multiple objects are presentalong the optical path, the use of a hybrid device can result inambiguities in associating distance returns with velocity returns.

FIG. 1 is a schematic illustration 100 of advantages of a hybrid lidarsensor that uses velocity-distance disambiguation, in accordance withsome implementations of the present disclosure. Depicted schematicallyis a hybrid lidar sensor 102 mounted on a section 104 (e.g., a roof) ofan autonomous vehicle (not shown) that may scan an outside environmentof the AV. Hybrid lidar sensor 102 can output a combined beam 106(depicted with a solid arrow) that can generate returns (reflections)from multiple objects, such as a stop sign 108, a speed limit sign 110,and a vehicle 112. The returns may be generated by reflecting surfacesthat are located close to the line of the combined beam 106, e.g., byreflecting surfaces 108(R), 110(R), and 112(R). As a result, threereflected signals 108(S), 110(S), and 112(S) may be generated (depictedwith dashed arrows), each carrying information about the velocities ofeach three respective reflecting surfaces 108(R), 110(R), and 112(R) anddistances to the three reflecting surfaces. Because each hybrid lidarcomponent processes a corresponding part of the reflected signals(pulsed or continuous) independently from processing of the other part,a hybrid lidar can be unable to disambiguate from differentdistance-velocity (often referred to in this disclosure as “L-V”)associations. In an instance where N velocities and N distances aredetected from N different objects, there could be N factorial (N!)possible pairings of these returns. For example, if three velocityreturns (1, 2, 3) and three distance returns (1′, 2′, 3′) are received,there could be 3 factorial (i.e. 3!=6) possible pairwise associations:1-1′, 2-2′, 3-3′; 1-1, 2-3′, 3-2′; 1-2′, 2-3′, 3-1′; etc.) Although FIG.1 depicts a situation of a combined lidar beam “skirting” variousobjects, in some instances, a combined lidar beam may pass through someof the objects. For example, a part of the combined lidar beam canreflect from a windshield of a first vehicle, while another part of thebeam passes through the windshield but reflects back from the rearwindow of the first vehicle. Yet another part of the beam can passthrough the rear window of the first vehicle and reflect from a secondvehicle (or some other object, e.g., a pedestrian, a road sign, abuilding, etc.).

Aspects and implementations of the present disclosure enable methods andsystems that achieve efficient distance-velocity disambiguation of thereceived (RX) signals that are reflected from objects in outsideenvironments and allow one to correctly associate distance returns of apulsed transmitted (TX) beam with returns of a continuous TX beam. Insome implementations, L-V disambiguation can be achieved by imparting aphase information (e.g., frequency or phase modulation) to thecontinuous beam output by the hybrid lidar. The phase information caninclude a number of markers sufficient for association of velocityreturns with a range of distances ΔL around a set of central distancesL₁, L₂, L₃ . . . . For example (as described in more detail below),based on a comparison of a phase information of a continuous RX signalwith a phase information of a LO copy of the TX beam, the distance to anobject having a detected velocity V_(A) can be coarsely determined to bewithin a range of [L₁−ΔL/2,L₁+ΔL/2]. Using a more accurate predictionobtained using pulsed RX returns, a return distance L_(B) within thisspecific range L_(B)∈[L₁−ΔL/2,L₁+ΔL/2] can be selected. A return pointcan then be identified as (L_(B), V_(A)). This return point (alone or inconjunction with other return points) can then be utilized for objectidentification using any of the known methods of clustering, iterativeclosest points (ICP) algorithms, and so on. In some implementations,instead of imparting a phase or frequency modulation to the continuousbeam, the continuous beam can be amplitude-modulated, with an amplitudeencoding providing similar markers to enable L-V disambiguation.

Advantages of the disclosed implementations include, on one hand,efficient disambiguation of distance and velocity sensing signalscompared with other hybrid lidars lacking such functionality. On theother hand, disclosed implementations have an advantage of simplicityand lower costs compared with FMCW lidars, which provide detaileddistance information based solely on the continuous beams. Frequency,phase, or amplitude modulation of the continuous beam in hybrid lidarscan be performed in a significantly coarser manner (than in FMCWlidars), as the continuous beam has only to identify a rough interval ofdistances. Such a coarser determination can be sufficient becauseadditional high-accuracy distance data is independently available frompulsed beam returns (wherein in an FMCW the continuous beam is also thesource of the distance information). Correspondingly, a hybrid lidarwith L-V disambiguation has lower demands (compared with FMCW lidars) tothe bandwidth for transmission of phase/frequency/amplitude informationthat is to be imparted in the continuous beam (as significantly fewerphase markers need to be transmitted and reliably detected). Similarly,a hybrid lidar has lower requirements for the accuracy of digital (e.g.,Fast Fourier Transform) processing as well as other electronicscomponents. Additional benefits of the disclosed implementations includesubstantial precision-to-cost advantages, decreased demands to accuracyof laser sources and/or optical modulators and the associatedelectronics.

FIG. 2 is a diagram illustrating components of an example autonomousvehicle (AV) 200 that uses one or more hybrid lidars withdistance-velocity disambiguation, in accordance with someimplementations of the present disclosure. Autonomous vehicles caninclude motor vehicles (cars, trucks, buses, motorcycles, all-terrainvehicles, recreational vehicle, any specialized farming or constructionvehicles, and the like), aircraft (planes, helicopters, drones, and thelike), naval vehicles (ships, boats, yachts, submarines, and the like),or any other self-propelled vehicles (e.g., robots, factory or warehouserobotic vehicles, sidewalk delivery robotic vehicles, etc.) capable ofbeing operated in a self-driving mode (without a human input or with areduced human input).

A driving environment 210 can be or include any portion of the outsideenvironment containing objects that can determine or affect how drivingof the AV occurs. More specifically, a driving environment 210 caninclude such objects (animate or inanimate) located outside the AV asroadways, buildings, trees, bushes, sidewalks, bridges, overpasses,underpasses, tunnels, construction zones, parking features, othervehicles, pedestrians, cyclists, and so on. The driving environment 210can be urban, suburban, rural, and so on. In some implementations, thedriving environment 210 can be an off-road environment (e.g. farming oragricultural land). In some implementations, the driving environment canbe an indoor environment, e.g., the environment of an industrial plant,a shipping warehouse, a hazardous area of a building, and so on. In someimplementations, the driving environment 210 can be substantially flat,with various objects moving parallel to a surface (e.g., parallel to thesurface of Earth). In other implementations, the driving environment canbe three-dimensional and can include objects that are capable of movingalong all three directions (e.g., balloons, leaves, etc.). Hereinafter,the term “driving environment” should be understood to include allenvironments in which motion of self-propelled vehicles can occur. Forexample, “driving environment” can include any possible flyingenvironment of an aircraft or a marine environment of a naval vessel.The objects of the driving environment 210 can be located at anydistance from the AV, from close distances of several feet (or less) toseveral miles (or more).

The example AV 200 can include a sensing system 220. The sensing system220 can include various electromagnetic (e.g., optical) andnon-electromagnetic (e.g., acoustic) sensing subsystems and/or devices.The terms “optical” and “light,” as referenced throughout thisdisclosure, are to be understood to encompass any electromagneticradiation (waves) that can be used in object sensing to facilitateautonomous driving, e.g., distance sensing, velocity sensing,acceleration sensing, rotational motion sensing, and so on. For example,“optical” sensing can utilize a range of light visible to a human eye(e.g., the 380 to 700 nm wavelength range), the UV range (below 380 nm),the infrared range (above 700 nm), the radio frequency range (above 1m), etc. In implementations, “optical” and “light” can include any othersuitable range of the electromagnetic spectrum.

The sensing system 220 can include a radar unit 226, which can be anysystem that utilizes radio or microwave frequency signals to senseobjects within the driving environment 210 of the AV 200. The radar unit226 can be configured to sense both the spatial locations of the objects(including their spatial dimensions) and their velocities (e.g., usingthe radar Doppler shift technology). The sensing system 220 can includea hybrid lidar 222 operating in accordance with implementations of thepresent disclosure. Hybrid lidar 222 can utilize wavelengths ofelectromagnetic waves that are shorter than the wavelength of the radiowaves and can, therefore, provide a higher spatial resolution andsensitivity compared with the radar unit 226. Hybrid lidar 222 caninclude a ToF system 223 (e.g., a lidar rangefinder), which can be alaser-based unit capable of determining distances to the objects in thedriving environment 210. Hybrid lidar 222 can further include a coherentsystem 224, which can use a continuous beam of light and opticalhomodyne or heterodyne detection for velocity determination. ToF system223 and coherent system 224 can share any number of optical componentsand devices (e.g., lenses, mirrors, apertures, diffractive opticalelements, beam splitters, optical amplifiers, and the like). ToF system223 and coherent system 224 can be configured to output beams along thesame optical path by combining the output beams into a single beam. Theoutput beams can nonetheless retain their uniqueness (e.g., by havingdifferent wavelengths, polarization, etc.) and upon reflection from atarget can be split (e.g., using beam splitters and/or diffractiveelements) and processed using separate optical and electronic processingcomponents. In some implementations, some of the processing (e.g.,digital processing) of the received beams can be performed by componentsthat are common for both ToF system 223 and coherent system 224.

Hybrid lidar 222 can further include L-V (distance-velocity)disambiguation 225, which should be understood as any number of physicalcomponents, elements, circuits, or software/firmware modules that enableassociation of distance returns with correct velocity returns (or viceversa). For example, L-V disambiguation 225 can include an opticalmodulator to impart angle modulation (e.g., phase and/or frequencymodulation) or amplitude modulation to a continuous beam output by thecoherent system 224. L-V disambiguation 225 can further include digital(or analog) processing that identifies modulation of RX signals,compares the identified modulation with the modulation of the LO copiesof TX beams, and determines possible distance ranges (intervals) forvarious objects that generate velocity returns.

ToF system 223 and/or coherent system 224 can include one or more lasersources producing and emitting signals and one or more detectors of thesignals reflected back from the objects. ToF system 223 and/or coherentsystem 224 can include spectral filters to filter out spuriouselectromagnetic waves having wavelengths (frequencies) that aredifferent from the wavelengths (frequencies) of the emitted signals. Insome implementations, ToF system 223 and/or coherent system 224 caninclude directional filters (e.g., apertures, diffraction gratings, andso on) to filter out electromagnetic waves that can arrive at thedetectors along directions different from the directions for the emittedsignals. ToF system 223 and/or coherent system 224 can use various otheroptical components (lenses, mirrors, gratings, optical films,interferometers, spectrometers, local oscillators, and the like) toenhance sensing capabilities of the sensors.

In some implementations, multiple hybrid lidars 222 can be mounted onthe same AV, e.g., at different locations separated in space, to provideadditional information about a transverse component of the velocity ofthe reflecting object. In some implementations, hybrid lidar(s) 222 canbe 360-degree scanning unit(s) in a horizontal direction. In someimplementations, hybrid lidar(s) 222 can be capable of spatial scanningalong both the horizontal and vertical directions. In someimplementations, the field of view can be up to 90 degrees in thevertical direction (e.g., with that at least a part of the region abovethe horizon can be scanned by the lidar signals or with at least part ofthe region below the horizon scanned by the lidar signals). In someimplementations, e.g., involving aeronautical applications, the field ofview can be a full sphere (consisting of two hemispheres). For brevityand conciseness, when a reference to “lidar technology,” “lidarsensing,” “lidar data,” and “lidar,” in general, is made in the presentdisclosure, such reference shall be understood also to encompass othersensing technology that operate at generally in the near-infraredwavelength, but may include sensing technology that operate at otherwavelengths, where applicable.

The sensing system 220 can further include one or more cameras 229 tocapture images of the driving environment 210. The images can betwo-dimensional projections of the driving environment 210 (or parts ofthe driving environment 210) onto a projecting plane of the cameras(flat or non-flat, e.g. fisheye cameras). Some of the cameras 229 of thesensing system 220 can be video cameras configured to capture acontinuous (or quasi-continuous) stream of images of the drivingenvironment 210. The sensing system 220 can also include one or moresonars 228, which can be ultrasonic sonars, in some implementations.

The sensing data obtained by the sensing system 220 can be processed bya data processing system 230 of AV 200. In some implementations, thedata processing system 230 can include a perception system 232.Perception system 232 can be configured to detect and track objects inthe driving environment 210 and to recognize/identify the detectedobjects. For example, the perception system 232 can analyze imagescaptured by the cameras 229 and can be capable of detecting trafficlight signals, road signs, roadway layouts (e.g., boundaries of trafficlanes, topologies of intersections, designations of parking places, andso on), presence of obstacles, and the like. The perception system 232can further receive the lidar sensing data to determine distances tovarious objects in the driving environment 210 and velocities (radialand transverse) of such objects. In some implementations, perceptionsystem 232 can use the lidar data in combination with the data capturedby the camera(s) 229. In one example, camera(s) 229 can detect an imageof road debris partially obstructing a traffic lane. Using the data fromthe camera(s) 229, perception system 232 can be capable of determiningthe angular extent of the debris. Using the lidar data, the perceptionsystem 232 can determine the distance from the debris to the AV and,therefore, by combining the distance information with the angular sizeof the debris, the perception system 232 can determine the lineardimensions of the debris as well.

In another implementation, using the lidar data, the perception system232 can determine how far a detected object is from the AV and canfurther determine the component of the object's velocity along thedirection of the AV's motion. Furthermore, using a series of quickimages obtained by the camera, the perception system 232 can alsodetermine the lateral velocity of the detected object in a directionperpendicular to the direction of the AV's motion. In someimplementations, the lateral velocity can be determined from the lidardata alone, for example, by recognizing an edge of the object (usinghorizontal scanning) and further determining how quickly the edge of theobject is moving in the lateral direction. The perception system 232 canreceive one or more sensor data frames from the sensing system 220. Eachof the sensor frames can include multiple points. Each point cancorrespond to a reflecting surface from which a signal emitted by thesensing system 220 (e.g., by hybrid lidar 222) is reflected. The typeand/or nature of the reflecting surface can be unknown. Each point canbe associated with various data, such as a timestamp of the frame,coordinates of the reflecting surface, radial velocity of the reflectingsurface, intensity of the reflected signal, and so on.

The perception system 232 can further receive information from apositioning subsystem, which may include a GPS transceiver (not shown),an inertial mechanical unit (IMU), and/or other systems configured toobtain information about the position of the AV relative to Earth andits surroundings. The positioning data processing module 234 can use thepositioning data, e.g., GPS and IMU data) in conjunction with thesensing data to help accurately determine location of the AV withrespect to fixed objects of the driving environment 210, such asroadways, lane boundaries, intersections, sidewalks, crosswalks, roadsigns, curbs, surrounding buildings, and so on, locations of which canbe provided by map information 235. In some implementations, the dataprocessing system 230 can receive non-electromagnetic data, such asaudio data (e.g., ultrasonic sensor data, or data from a mic picking upemergency vehicle sirens), temperature sensor data, humidity sensordata, pressure sensor data, meteorological data (e.g., wind speed anddirection, precipitation data), and the like.

Data processing system 230 can further include an environment monitoringand prediction component 236, which can monitor how the drivingenvironment 210 evolves with time, e.g., by keeping track of thelocations and velocities of the animated objects (relative to Earth). Insome implementations, environment monitoring and prediction component236 can keep track of the changing appearance of the driving environmentdue to motion of the AV relative to the driving environment. In someimplementations, environment monitoring and prediction component 236 canmake predictions about how various animated objects of the drivingenvironment 210 will be positioned within a prediction time horizon. Thepredictions can be based on the current locations and velocities of theanimated objects as well as on the tracked dynamics of the animatedobjects during a certain (e.g., predetermined) period of time. Forexample, based on stored data for object 1 indicating accelerated motionof object 1 during the previous 3-second period of time, environmentmonitoring and prediction component 236 can conclude that object 1 isresuming its motion from a stop sign or a red traffic light signal.Accordingly, environment monitoring and prediction component 236 canpredict, given the layout of the roadway and presence of other vehicles,where object 1 is likely to be within the next 3 or 5 seconds of motion.As another example, based on stored data for object 2 indicatingdecelerated motion of object 2 during the previous 2-second period oftime, environment monitoring and prediction component 236 can concludethat object 2 is stopping at a stop sign or at a red traffic lightsignal. Accordingly, environment monitoring and prediction component 236can predict where object 2 is likely to be within the next 1 or 3seconds. Environment monitoring and prediction component 236 can performperiodic checks of the accuracy of its predictions and modify thepredictions based on new data obtained from the sensing system 220.

The data generated by the perception system 232, the GPS data processingmodule 234, and environment monitoring and prediction component 236 canbe used by an autonomous driving system, such as AV control system(AVCS) 240. The AVCS 240 can include one or more algorithms that controlhow AV is to behave in various driving situations and drivingenvironments. For example, the AVCS 240 can include a navigation systemfor determining a global driving route to a destination point. The AVCS240 can also include a driving path selection system for selecting aparticular path through the immediate driving environment, which caninclude selecting a traffic lane, negotiating a traffic congestion,choosing a place to make a U-turn, selecting a trajectory for a parkingmaneuver, and so on. The AVCS 240 can also include an obstacle avoidancesystem for safe avoidance of various obstructions (rocks, stalledvehicles, a jaywalking pedestrian, and so on) within the drivingenvironment of the AV. The obstacle avoidance system can be configuredto evaluate the size of the obstacles and the trajectories of theobstacles (if obstacles are animated) and select an optimal drivingstrategy (e.g., braking, steering, accelerating, etc.) for avoiding theobstacles.

Algorithms and modules of AVCS 240 can generate instructions for varioussystems and components of the vehicle, such as the powertrain, brakes,and steering 250, vehicle electronics 260, signaling 270, and othersystems and components not explicitly shown in FIG. 2 . The powertrain,brakes, and steering 250 can include an engine (internal combustionengine, electric engine, and so on), transmission, differentials, axles,wheels, steering mechanism, and other systems. The vehicle electronics260 can include an on-board computer, engine management, ignition,communication systems, carputers, telematics, in-car entertainmentsystems, and other systems and components. The signaling 270 can includehigh and low headlights, stopping lights, turning and backing lights,horns and alarms, inside lighting system, dashboard notification system,passenger notification system, radio and wireless network transmissionsystems, and so on. Some of the instructions outputted by the AVCS 240can be delivered directly to the powertrain, brakes, and steering 250(or signaling 270) whereas other instructions outputted by the AVCS 240are first delivered to the vehicle electronics 260, which generatecommands to the powertrain and steering 250 and/or signaling 270.

In one example, the AVCS 240 can determine that an obstacle identifiedby the data processing system 230 is to be avoided by decelerating thevehicle until a safe speed is reached, followed by steering the vehiclearound the obstacle. The AVCS 240 can output instructions to thepowertrain, brakes, and steering 250 (directly or via the vehicleelectronics 260) to 1) reduce, by modifying the throttle settings, aflow of fuel to the engine to decrease the engine rpm, 2) downshift, viaan automatic transmission, the drivetrain into a lower gear, 3) engage abrake unit to reduce (while acting in concert with the engine and thetransmission) the vehicle's speed until a safe speed is reached, and 4)perform, using a power steering mechanism, a steering maneuver until theobstacle is safely bypassed. Subsequently, the AVCS 240 can outputinstructions to the powertrain, brakes, and steering 250 to resume theprevious speed settings of the vehicle.

The “autonomous vehicle” can include motor vehicles (cars, trucks,buses, motorcycles, all-terrain vehicles, recreational vehicle, anyspecialized farming or construction vehicles, and the like), aircrafts(planes, helicopters, drones, and the like), naval vehicles (ships,boats, yachts, submarines, and the like), robotic vehicles (e.g.,factory, warehouse, sidewalk delivery robots), or any otherself-propelled vehicles capable of being operated in a self-driving mode(without a human input or with a reduced human input). “Objects” caninclude any entity, item, device, body, or article (animate orinanimate) located outside the autonomous vehicle, such as roadways,buildings, trees, bushes, sidewalks, bridges, mountains, other vehicles,piers, banks, landing strips, animals, birds, or other things.

FIG. 3 is a block diagram illustrating an example implementation of ahybrid lidar 300 (e.g., hybrid lidar 222 of FIG. 2 ) that usesdistance-velocity disambiguation, in accordance with someimplementations of the present disclosure. Hybrid lidar 300 can includemultiple light sources, such as a pulsed light source 302 and acontinuous light source 304. Each light source is configured to produceone or more beams of light. “Beams” should be understood herein to referto any signals of electromagnetic radiation, such as beams, wavepackets, pulses, sequences of pulses, or other types of signals. Pulsedlight source 302 and/or continuous light source 304 can use a broadbandlaser, a narrow-band laser, a light-emitting diode, a Gunn diode, andthe like. Any of lasers utilized by pulsed light source 302 and/orcontinuous light source 304 can be a semiconductor laser, a gas laser,an ND:YAG laser, or any other type of a laser. Pulsed light sources 302can utilize a single-pulse laser, a repetitively pulsed laser, and thelike. Pulsed light sources 302 can be synchronized with a scanningmechanism (e.g., actuator) of the lidar transmitter. More specifically,pulsed light source 302 can generate one or any other number of pulsesfor each direction of scanning. Pulses generated by pulsed light source302 can be monochromatic pulses a having central carrier frequency λ₁ inany suitable optical range (e.g., infrared) and a pulse duration thatcan be substantially greater than the period of the light oscillations,λ₁/c, but still substantially smaller than the time of flight to atypical target, L/c, where L is the distance to the target and c is thespeed of light. Continuous light source 304 can produce a beam whoseduration is substantially greater than the duration of pulses (and/orthe time of flight to a typical target). The beam produced by continuouslights source 304 (herein often referred to as a continuous beam) canhave a frequency λ₂ that is different from frequency λ₁. The term“continuous” should not be understood as an indication that the producedbeam is always turned on. Instead, “continuous” indicates that thebeam's duration can be, in some implementations, longer than the time offlight to a typical target, L/c.

In some implementations, pulses output by pulsed light source 302 and/orcontinuous beams output by light source 304 can be conditioned(pre-processed) by one or more components or elements of a beampreparation stages 310 and 312. Preprocessing can ensure a narrow-bandspectrum, target linewidth, coherence, polarization (e.g., circular orlinear), and other optical properties that enable measurements (e.g.,coherent Doppler measurements) described below. Beam preparation can beperformed using filters (e.g., narrow-band filters), resonators (e.g.,resonator cavities, crystal resonators, etc.), polarizers, feedbackloops, lenses, mirrors, diffraction optical elements, and other opticaldevices. For example, if the pulsed light source 302 is a broadbandlight source, the output light can be filtered to produce a narrowbandbeam. In some implementations, where pulsed light source 302 andcontinuous light source 304 produce beams that have a desired linewidthand coherence, the beams can still be additionally filtered, focused,collimated, diffracted, amplified, polarized, etc., to produce one ormore beams of a desired spatial profile, spectrum, duration, frequency,polarization, repetition rate, and so on.

In some implementations, light output by pulsed light source 302 (and,in some implementations, continuous light source 304) can beadditionally processed by a preamplifier 314. Preamplifier 314 caninclude (not shown) an additional pump laser, a combiner, and an opticalamplifier. In some implementations, the pump laser is a single mode pumpdiode. In some implementations, the combiner of preamplifier 314 is awavelength division multiplexer that combines a pulsed optical signalgenerated by pulsed light source 302 with a signal generated by the pumplaser. In some implementations, the amplifier of pulsed light source 302is an erbium-doped single-mode fiber. In some implementations, theamplifier of pulsed light source 302 is an erbium/ytterbium-dopeddispersion-compensating fiber (Er/Yb-DCF). In some implementations, theamplifier may be a semiconductor optical amplifier (which may be furtherimplemented as part of a photonic integrated circuit).

The light output by beam preparation stage 312 can be inputted into anoptical modulator 320 to provide modulation to the continuous beamoutputted by beam preparation stage 312. “Optical modulation” is to beunderstood herein as referring to any form of angle modulation, such asphase modulation (e.g., any temporal sequence of phase changes Δϕ_(j)added to the phase of the beam), frequency modulation (e.g., anysequence of frequency changes Δf_(j), either positive or negative),amplitude modulation, or any other type of modulation (e.g., acombination of phase and frequency modulation) that affects the phase ofthe wave. Amplitude modulation can be imparted, e.g., using a poweramplifier that increases or reduces (possibly, down to zero amplitude)the amplitude of the continuous beam. Amplitude modulation can beapplied to the light in combination with angle modulation or separately,without angle modulation. In some implementations, optical modulator 320can be or include an acousto-optic modulator, an electro-opticmodulator, a Lithium Niobate modulator, a heat-driven modulator, aMach-Zender modulator, and the like, or any combination thereof.

In various implementations, phase shifts (and, similarly, frequency oramplitude changes) can have any number of values, e.g., N discrete phasevalues across the phase interval 2π. A temporal sequence of phase shiftsΔϕ_(j) (phase encoding) can be added by a radio frequency (RF) source(or any other suitable source) outputting a signal (e.g., an RFelectrical signal) to optical modulator 320. In some implementations,the RF signals applied to optical modulator 320 can cause opticalmodulator 320 to impart consecutive phase shifts Δϕ_(j) to thecontinuous light beam. In some implementations, the RF signals appliedto optical modulator 320 can cause optical modulator 320 to impartfrequency shifts Δf_(j) to the continuous light beam, e.g., a sequenceof up-chirps interspersed with down-chirps. In some implementations, apower amplifier controlled by RF signals can impart amplitude changesΔA_(j) to the continuous light beam. In one exemplary implementation, aperiod of phase or frequency encoding can be 2 μs with 10 differentphase, frequency, or amplitude values of 0.2 μs duration used withineach 2 μs period. As described below in relation to FIG. 5 and FIGS.6A-B, the resolution of such an encoding would be ΔL=60 m over eachinterval L₀=300 m (referred herein to as the maximum unambiguous rangeintervals), meaning that this phase/frequency encoding would be capableof distinguishing target objects located at distances 30 m<L<90 m fromobjects located at distances 90 m<L<150 m. At the same time, objectsthat are located at 125 m may provide a response that is similar to aresponse of an object located at 425 m (125 m +300 m). Disambiguation ofobjects located at distances that differ by an integer number of maximumunambiguous range intervals L₀ may then be enabled based on theintensity of reflected signals. Increasing the period of phase/frequencyencoding increases the maximum unambiguous range interval L₀ whereasincreasing the number of different phase/frequency values within theperiod of encoding increases resolution (decreases ΔL).

After optical modulation, light output by optical modulator 320 canundergo spatial separation at a beam splitter (not depicted) to splitoff (dashed line) a local oscillator (LO) copy 330 of the modulatedcontinuous beam. LO copy 330 can be used as a reference signal to whicha signal reflected from a target can be compared. The beam splitter canbe (or include) a prism-based beam splitter, a partially-reflectingmirror, a polarizing beam splitter, a beam sampler, a fiber opticalcoupler (optical fiber adaptor), or any similar beam splitting element(or a combination of two or more beam-splitting elements). The lightbeam can be delivered to the beam splitter (as well as between any othercomponents depicted in FIG. 3 ) over air or over light carriers, such asoptical fibers or other types of waveguide devices.

The signal copy of the light beam (solid line) can be delivered tooptical combiner 340 to be combined with the pulsed beam output bypreamplifier 314. Combining of the two beams can be performed to ensurethat both beams follow the same optical path and, therefore, are outputtowards the same target. Even though, for brevity and conciseness, onlyLO copy 330 for the continuous beam is depicted in FIG. 3 , it should beunderstood that a local oscillator copy of the pulsed beam can similarlybe retained on the hybrid lidar 300 as a reference signal for subsequentprocessing of the reflected pulses.

The combined beam output by optical combiner 340 can be amplified byamplifier 350 before being outputted, through a transmission (TX)optical interface 360, as a TX beam 362 towards one or more objects 364,which can be objects in the driving environment 210 of FIG. 2 . Opticalinterface 360 can include an aperture and a combination of opticalelements, e.g., lenses, mirrors, collimators, polarizers, waveguides,and the like. Optical elements of TX optical interface 360 can be usedto direct TX beam 362 to a desired region in the outside environment.Output TX beam 362 can travel to one or more objects 364 and, uponinteraction with the respective objects, generate reflected beams (alsoreferred to as RX signals) 366, which can enter hybrid lidar 300 via areceiving (RX) optical interface 368. In some implementations, RXoptical interface 368 can share at least some optical elements with theTX optical interface 360, e.g., aperture, lenses, mirrors, collimators,polarizers, waveguides, and so on.

RX signals 366 can include both the pulsed signals and continuoussignals reflected by objects 364. In some instances, RX signals 366 caninclude multiple returns (reflections) of the pulsed beam and multiplereturns of the continuous beam. Multiple returns can be caused bymultiple objects reflecting the same TX beam 362. For example, m returns(herein referred to as distance returns) of the pulsed beam and nreturns of the continuous beam (herein referred to as velocity returns)can be received as part of RX signals 366. To associate at least some ofthe distance returns with velocity returns (or vice versa), hybrid lidar300 can process such returns separately. Specifically, hybrid lidar 300can include a beam splitter 370 capable of spatially separating distancereturns from velocity returns. In some implementations, distance returnscan have wavelength λ₁ that is different from wavelength of velocityreturns λ₂. In such implementations, beam splitter 370 can include oneor more optical elements that are sensitive to the wavelength of light,including diffraction optical elements, prisms having afrequency-dependent refractive index, wavelength-sensitive opticalcirculators, wavelength-division multiplexers, wavelength divisioncouplers (e.g., fiber optic couplers), or any other dispersive opticalelements capable of separating and directing light of different spectralcontent (e.g., wavelengths) along different optical paths. For example,RX signals associated with distance returns (wavelength λ₁) can bedirected towards intensity detection module 380 whereas RX signalsassociated with velocity returns (wavelength λ₂) can be directed towardscoherent detection module 382. Intensity detection module 380 canconvert optical intensity of the RX signals to electrical signals, e.g.,using photoelectric circuit elements (e.g., photodiodes) and provideelectrical signals to an analog-to-digital converter (ADC) 384 fordigitizing the intensity of the received distance returns I_(RX)(t). Aprocessing device 390 can then determine the shift (delay) in time ofthe distance returns compared with the intensity of the transmittedpulsed beam intensity I_(TX)(t) (which can be available via a LO copy ofthe transmitted pulsed beam). Based on the time delay t_(delay) of amaximum of I_(RX)(t) compared with a maximum of I_(TX)(t), the distanceto the reflecting object 364 can be determined as L=ct_(delay)/2.(Factor ½ accounting for the fact that the signal travels the distance Lto the object twice, before and after the reflection).

Coherent detection module 382 can include one or more filters to beapplied to coherent components of RX signals 366. Filters can includematched filters (in the instances where phase modulation is beingdetected), frequency filters (when frequency modulation is beingdetected), amplitude/peak filters (when amplitude modulation is beingdetected), or any combination thereof. Coherent detection module 382 canprocess the velocity returns (wavelength λ₂) using one or more balancedphotodetectors to detect phase information carried by the coherentcomponent of RX signals 366. A balanced photodetector can havephotodiodes connected in series and can generate ac electrical signalsthat are proportional to a difference of input optical modes (which canadditionally be processed and amplified). A balanced photodetector caninclude photodiodes that are Si-based, InGaAs-based, Ge-based,Si-on-Ge-based, and the like. In some implementations, balancedphotodetectors can be manufactured on a single chip, e.g., usingcomplementary metal-oxide-semiconductor (CMOS) structures, siliconphotomultiplier (SiPM) devices, or similar systems. Balancedphotodetector(s) can also receive LO copy 330 of the continuous TX beam.A balanced photodetector can detect a phase difference between phases ofthe continuous TX beam and can output electrical signals representativeof the information about relative phases of the RX signal and thecontinuous TX beam. The electrical signal output by coherent detectionmodule 382 can be digitized by ADC 384 and processed by processingdevice 390 for distance-velocity disambiguation using L-V disambiguationmodule 125. Processing by processing device 390 can include usingspectral analyzers, such as Fast Fourier Transform (FTT) analyzers,digital filters, mixers, local RF oscillators (e.g., carryinginformation about phase/frequency modulation imparted to the continuousbeam by optical modulator 320), and any other suitable components and/ordevices.

In some implementations, L-V disambiguation module 125 can obtain ndistance values L₁, L₂, . . . L_(n) for n distance returns and comparethose distance returns with m velocity returns (V₁;[L_(1min),L_(1max)]),(V₂;[L_(2min),L_(2max)]), . . . (V_(m);[L_(m min),L_(m max)]), that, inaddition to (radial) velocities V_(j), identify approximate (coarse)ranges of distances [L_(j min),L_(j max)] to the reflecting objectsassociated with the respective returns. Having identified to whichranges of distances various distance values V_(k) belong, L-Vdisambiguation module 125 of the processing device 390 can identifyaccurate distance-velocity associations (L_(k);V_(j)). For example, ifL-V disambiguation module 125 identifies that L₁∈[L_(3min),L_(3max)], afirst distance-velocity association can be identify as (L₁;V₃). Otherassociations can be determined in a similar manner.

FIG. 4A, FIG. 4B, and FIG. 4C illustrate example implementations offrequency encodings that can be used to modulate a continuous beamoutput by a hybrid lidar, in accordance with some implementations of thepresent disclosure. Though the frequency encodings herein areillustrated for concreteness, it should be understood that othersubstantially similar encodings can be used for phase or amplitudemodulation of the continuous beam. FIG. 4A illustrates a symmetric“staircase” frequency modulation 400 that uses eight different frequencyvalues 0, Δf, 2Δf, . . . 7Δf (e.g., counted from a base value f₀ set bycontinuous light source 304 of FIG. 3 ), each implemented for aparticular time interval τ. The modulation shown has a period 15τ and isrepeated multiple times. Any other number N of frequency values can beused instead, for example, four, ten, or any other number of frequencyvalues. In some implementations, duration of at least some of theintervals can be different from τ. FIG. 4B illustrates an asymmetricstaircase frequency modulation 410 that uses fifteen different frequencyvalues

0,Δf, 2Δf, . . . 6Δf, 7Δf, 5.5Δf, 4.5Δf, . . . 0.5Δf, −0.5Δf,

each implemented for a specific time interval τ. The modulation shownhas a period 16τ and can be repeated multiple times. The advantage ofthe modulation illustrated in FIG. 4B is that no two values within oneperiod are the same, which (as described below in relation to FIGS. 5,6A-B can be advantageous for improving distance resolution). Any othernumber N of frequency changes can be used instead. In someimplementations, duration of at least some of the intervals can bedifferent from τ. FIG. 4C illustrates a directional staircase 420 withresets, in which frequency is monotonically increased from a minimumfrequency (e.g., −2Δf) to a maximum frequency (e.g., 2Δf) followed by areset back to the minimum frequency. In some implementations, adirectional staircase can have frequencies changing in the oppositedirection, e.g., from a maximum to a minimum, with a reset back to themaximum frequency. In those implementations where N increasing (ordecreasing) frequency values are used, one period of frequencymodulation can be Nτ, if all time intervals have the same duration τ.The advantage of the modulation illustrated in FIG. 4C (as explainedbelow in relation to FIGS. 6A-B) is that the same detected frequencychanges can be indicative of the same time delays regardless of whereexactly such frequency changes occur on the staircase. In someimplementations, duration of at least some of the intervals can bedifferent from τ. The example frequency (and, similarly, phase oramplitude) encodings are presented by way of illustration and not by wayof limitation. It should be understood that practically numerous othervarious modulation sequences (e.g., as can be permitted by thebandwidth, bit-depth, and resolution of the electronics circuitry beingused) can be used with the number of different frequency/phase/amplitudeintervals and the duration of such intervals can be determined by atarget accuracy of the distance disambiguation.

FIG. 5 illustrates identification 500 of a range of distances by ahybrid lidar system that uses an example frequency encoding of acontinuous beam, in accordance with some implementations of the presentdisclosure. Depicted by a black dot is a frequency f_(RX)(t) of thereflected RX signal generated upon reflection of the continuous beamfrom a particular target and arriving at the hybrid lidar at time t.Also depicted is an example staircase of the LO copy of the transmittedoutput beam. At the time of detection t, the LO copy has a frequencyvalue f_(LO)(t) that is within a k-th plateau, f_(k). Using coherentdetection module 382 (e.g., balanced photodetectors), the hybrid lidarcan identify the beating frequencyf_(BEAT)=f_(LO)(t)−f_(RX)(t)=f_(k)(t)−_(RX)(t) and determine that thebeating frequency points to a j-th plateau of the LO. The j-th plateauis, therefore, associated with a time in the past t₁=t−2L/c (depictedwith a white dot) when the signal was transmitted to the object. Notethat the detected frequency f_(RX)(t) can be different from thefrequency of the transmitted beam, f_(j), with the difference amountingto the Doppler shift, f_(RX)(t)−f_(j)=2Vf_(j)/c=2V/λ₁, caused by themotion of the object with a radial velocity V (V>0 if the object ismoving towards the lidar, and V<0 if the object is moving away). Inorder for the hybrid lidar to identify the association of the detectedfrequency f_(RX)(t) with a correct plateau f_(j), the frequency step Δfbetween different (e.g., adjacent) plateaus can be set to be at leasttwice (and in some implementations, substantially more) than a typicalexpected Doppler shift. For example, if the maximum expected velocity ofan object in a driving environment is 150 mph (V_(max)=67 m/s), thefrequency step can be Δf>4V_(max)/λ_(j). For an example infrared lidarof wavelength λ_(j)=900 nm, this means Δf>0.3 MHz.

Processing of continuous signal returns can include some or all of thefollowing. A beating frequency f_(BEAT) is first determined using thecurrent LO plateau f_(k). The beating frequency is then used to identifya past plateau f_(j) associated with the TX signal, as the plateau j forwhich the difference f_(k)−f_(BEAT)−f_(j) is the smallest. Next, thisdifference (attributed to the Doppler shift) is used to determine thevelocity V=λ_(J)(f_(k)−f_(BEAT)−f_(j))/2 of the reflecting object. Inmany implementations, since Δf<<f₀, the wavelength λ_(j) can beapproximated with the wavelength of the unmodulated signal λ_(j)≈λ₀while still ensuring an excellent accuracy. Finally, a range of possibledistances L_(MIN)≤L≤L_(MAX) can be identified based on the relativepositions (within the LO staircase) of the current LO plateau(“detection plateau”) f_(k) and the past plateau f_(j) associated withthe TX beam (“TX plateau”) f_(j). Namely, there can be an uncertainty ofwhere exactly the current moment of time t is within the duration of thedetection plateau f_(k), and a similar uncertainty where exactly themoment of beam transmission t₁=t−2L/c is within the duration of the pastTX plateau f_(j). As a result, twice the minimum distance to thereflecting object, 2L_(MIN) is the time between the end of the TXplateau f_(j) and the start of the detection plateau f_(k) (as depictedin FIG. 5 ). Similarly, twice the maximum distance to the reflectingobject, 2L_(MAX) is the time between the start of the TX plateau f_(j)and the end of the detection plateau f_(k) (as also depicted in FIG. 5). Having identified the range of possible distances based on thecontinuous RX signal, the processing device of the hybrid lidar canidentify a distance return within this range L_(MIN)≤L≤L_(MAX) and,therefore, identify the exact distance to the reflecting object.

FIG. 6A and FIG. 6B further illustrate identification of a range ofdistances by a hybrid lidar system using an example directionalfrequency staircase encoding, in accordance with some implementations ofthe present disclosure. FIG. 6A illustrates the frequency encoding 600of FIG. 4C, which uses five different values of frequency 0, ±Δf, ±2Δf(e.g., counted from a reference frequency, which can be a frequency ofthe unmodulated continuous light source 304 of FIG. 3 ). Five frequencyplateaus are used as an illustration only. It should be understood thatany other number N of plateaus (e.g., seven, ten, twenty, etc.) can beused in a similar fashion, with larger number N allowing fordetermination of the velocity ranges with a higher resolution. Arrowsconnecting different plateaus indicate various beating frequencies thatcan be detected by coherent photodetection module of the hybrid lidar(with smaller Doppler shifts not indicated for conciseness). The arrowsshould be understood as starting at the detection plateaus f_(k)(plateaus of the LO copy at the moment of detection of the RX signals,as described in conjunction with FIG. 5 ) and ending at the TX plateausf_(j) (plateaus of the LO at a prior moment of beam transmission).Accordingly, each arrow points back in time, from the moment ofdetection to the moment of transmission. Each arrow has an indication ofthe beating frequency f_(BEAT)=f_(k)−f_(j) ascribed to it. For example,a dotted arrow with the beating frequency f_(BEAT)=0 is shown connectingtwo plateaus of equal frequencies (e.g., two 2Δf plateaus); dashedarrows indicate negative beating frequencies; dash-dotted arrowsindicate positive beating frequencies. The directional staircase withresets, illustrated in FIG. 6A, possesses the property that the samebeating frequencies f_(k)−f_(j) connect plateaus separated by the samenumber of time intervals τ (plateau durations) regardless of thespecific end values f_(k) and f_(j). For example, a zero beatingfrequency indicates the same plateau (a fast lidar return where the timeof flight is at most τ) or plateaus that are separated by fourintervening plateaus (time of flight between 4τ and 5τ). Similarly, abeating frequency of +Δf (or −4Δf) indicates the time of flight between3τ and Sτ; a beating frequency of +2Δf (or −3Δf) indicates the time offlight between 2τ and 4τ; a beating frequency of +3Δf (or −2Δf)indicates the time of flight between τ and 3τ; and a beating frequencyof +4Δf (or −Δf) indicates the time of flight between 0 and 2τ.

FIG. 6B illustrates identification 610 of the ranges of distances basedon beating frequencies by a hybrid lidar that uses the directionalfrequency staircase encoding of FIG. 6A. Because the total time offlight of the transmitted and reflected signals multiplied by the speedof light c is twice the distance to the reflecting object, 2L, detectionof a specific beating frequency confines the value 2L to a particular(known) range of length 2 cτ or, equivalently, to a known range oflength cτ for the distance L itself. FIG. 6B shows association ofdifferent beating frequencies with various distances. Only one period ofdistance associations is shown as the interval [0, 5 cτ] is understoodto be repeated (with the same associated beating frequencies)periodically to larger distances with the period of distance L₀=5 cτ.More specifically, the beating frequency +3Δf is associated with theinterval [cτ, 3 cτ] as well as with the intervals [6 cτ, 8 cτ], [11 cτ,13 cτ], and so on. In some instances, only one of the RX pulsed signalsmay have a distance identified to be within any of those intervals. Insuch instances, L-V disambiguation of the respective pulsed return iscomplete. In other instances, multiple pulsed returns may be withinintervals associated with a specific beating frequency. For example, onepulsed return can be within the interval [cτ, 3 cτ] while another pulsedreturn can be within the interval [6 cτ, 8 cτ]. Further disambiguationof such returns can be performed using the intensity of the returns,since intensity of reflection from an object located at distances cτ≤L≤3cτ can be significantly weaker than the intensity of a return from anobject located at distances 6 cτ≤L≤8 cτ.

In other implementations, any different number N≠5 of steps in thedirectional staircase can be used, with various returns still identifiedwithin intervals of length 2 cτ up to the addition of an integer numberof distances L₀=Ncτ/2. For example, if τ=0.2 μs, and N=10, theresolution would be approximately cτ=60 m up to the addition ofL₀=Ncτ/2=300 m (or 600 m, 900 m, etc.). Disambiguation of objectslocated at distances that differ by (an integer number of) L₀ can thenbe enabled using sensing data obtained with pulsed beams and/orintensity data. Increasing the period of phase/frequency encodingincreases the resolution distance L₀ whereas increasing the number ofdifferent phase/frequency values within the period of encoding increasesaccuracy of disambiguation (decreases ΔL).

Although FIGS. 5 and 6A-B describe disambiguation based on frequencymodulation, similar techniques can be deployed using phase or amplitudemodulation. For example, a TX beam (and, therefore, its LO copy), may beproportional to cos[f₀t+ϕ(t)], where ϕ(t) is a phase encoding, which canbe of any form shown in FIG. 5 (or any other suitable encoding). Adifference between the phase of the LO copy of the TX beam, f₀t+ϕ(t), atthe time of detection t can be compared (e.g., using a balancedphotodetector) with phase (f₀+f_(D))t₁+ϕ(t₁) of the TX beam at anearlier moment of transmission t₁=t−2L/c. The velocity of the reflectingobject can be determined from the Doppler shift f_(D), as describedabove. Phase ϕ(t) can be identified as belonging to, e.g., k-th interval(plateau) whereas phase ϕ(t₁) can be identified as belonging to adifferent (or, in some instances, the same) j-th interval (plateau).Correspondingly, the difference between the times t₁−t=2L/c can bedetermined as described above based on the values k and j to within anaccuracy Δt set by the duration of various intervals (plateaus) τ.Finally, based on the determined time difference, a processing device ofthe hybrid laser can determine the range of values [L_(MIN), L_(MAX)] ofthe distance to the object L. It should be noted that the range ofvalues can include multiple intervals that are separated by the maximumunambiguous range interval L₀, e.g., [L_(MIN), L_(MAX)], [L₀+L_(MIN),L₀+L_(MAX)], [2L₀+L_(MIN), 2L₀+L_(MAX)], etc.

FIG. 7 and FIG. 8 depict flow diagram of example methods 700 and 800 ofusing a hybrid lidar for range and velocity detection, in accordancewith some implementations of the present disclosure. Methods 700 and 800can be performed using systems and components described in relation toFIGS. 1-6 , e.g., hybrid lidar 222 of the sensing system 220 of anautonomous vehicle. Methods 700 and 800 can be performed as part ofobtaining a point cloud for sensing various objects in a drivingenvironment of the autonomous vehicle. Various operations of methods 700and 800 can be performed in a different order compared with the ordershown in FIGS. 7-8 . Some operations of methods 700 and 800 can beperformed concurrently with other operations. Some operations can beoptional. Methods 700 and 800 can be used for determination of range andvelocity of objects in outside environments of autonomous vehicles,including driving environments. Methods 700 and 800 can be used toimprove coverage, resolution, and speed of detection of objects andtheir state of motion, as well as decrease costs, size, and complexityof optical sensing systems.

FIG. 7 depicts a flow diagram of an example method 700 ofdistance-velocity disambiguation in hybrid lidars, in accordance withsome implementations of the present disclosure. Method 700 can includeproducing, at block 710, a first beam comprising one or more pulses andfurther producing, at block 720, a second beam. It should be understoodthat the terms “first” and “second” are used herein as mere identifiersand do not presuppose any specific ordering or a temporal/spatialrelationship. The second beam can be a continuous beam. Any of the firstbeam and the second beam can be a coherent beam, e.g., a laser beam. Thefirst beam and/or the second beam can be prepared (e.g., filtered,collimated, polarized, etc.) to produce the light beam of desiredproperties. For example, the spectrum of the first beam and/or thesecond beam can be narrowed so that a target linewidth is achieved. Insome implementations, for the purpose of separating received signalsgenerated by the first and the second beams, the first beam can have afirst spectral content and the second beam can have a second spectralcontent that is different from the first spectral content. For example,the first beam can have a first central wavelength and the second beamcan have a second central wavelength that is different from the firstcentral wavelength of the first beam.

At block 730, method 700 can continue with imparting a modulation to thesecond beam. In some implementations, the modulation imparted to thesecond beam can be an angle modulation that includes a frequencymodulation and/or a phase modulation. For example, the angle modulationcan include at least one of (i) a temporal sequence of frequency shifts,or (ii) a temporal sequence of phase shifts. In some implementations,the phase information (e.g., angle modulation) imparted to the secondbeam can include at least four different frequency shifts or fourdifferent phase shifts. The phase information can be imparted by anoptical modulator, such as an acousto-optic modulator or anelectro-optic modulator. For example, the phase information can beimparted by an acoustic wave induced in the acousto-optic modulator. Theacoustic wave can have a frequency that is an integer number of afrequency shift Δf. In some implementations, the modulation imparted tothe second beam can be an amplitude modulation. In some implementations,phase information can be imparted by a phase modulator, such as aMach-Zehnder modulator, or other suitable devices.

At block 740, method 700 can continue with transmitting the first beamand the second beam to an outside environment. In some implementations,the first beam and the second beam can be combined and transmitted alonga same optical path. At block 750, method 700 can continue withreceiving, from the outside environment, a plurality of received (RX)signals caused by at least one of the first beam or the second beam. Forexample, the first beam and the second beam can strike two or moreobjects (e.g., located along or near the same line of sight) and,correspondingly, generate reflected RX signals from the two or moresignals. More specifically, the first beam can be used to identifyaccurate distances to the two or more objects (e.g., by measuring thetime of flight of the respective pulsed signals to and from theobjects). The second beam can be used to identify the velocities of theobjects (e.g., by detecting the Doppler shifts of the continuoussignals). At block 760, method 700 can continue with disambiguating thedetected distance returns and the velocity returns. More specifically,the hybrid lidar can determine that a first RX signal of the pluralityof RX signals and a second RX signal of the plurality of RX signals arereflected by the same object. The first RX signal can be representativeof a distance L to the object and the second RX signal is representativeof the velocity V of the object. Additionally, the second RX signal canbe further representative of an interval of possible distances to theobject [L_(MIN),L_(MAX)], the interval of possible distances beingidentified based on the modulation of the second RX signal. For example,the hybrid lidar can include a coherent photodetector configuration togenerate an electronic signal representative of a difference between thephase information of the second RX signal and the phase information ofthe second beam. More specifically, the coherent photodetector can havea first optical input and a second optical input. A coherentphotodetector can be an optical system that includes beam splitters,polarizers, optical amplifiers, and one or more balanced photodetectors.Each balanced photodetector can include one or more pairs of photodiodesconnected in series and configured to generate ac electrical signalsthat are proportional to a difference of optical modes input into thephotodiodes. Balanced photodetectors can include Si-based, InGaAs-based,Ge-based, Si-on-Ge-based, etc., photodiodes, SiPM diodes, and/or anyother suitable devices, which can further be integrated on a singlechip, such as a CMOS chip. The first optical input into the coherentphotodetector can be the second RX signal (e.g., the continuous part ofthe reflected beam). The second optical input into the coherentphotodetector can be a local oscillator copy of the second beam (e.g.,the continuous beam transmitted to the target). At block 770, method 700can continue with associating the distance L to the object, determinedfrom the first RX signal, with the velocity of the object V determinedfrom the second RX signal, based on L being within the interval[L_(MIN),L_(MAX)] of possible distances to the object, as determinedfrom the second RX signal.

FIG. 8 depicts a flow diagram of an example method 800 of associatingranges of distances to reflecting objects using returns generated by acontinuous beam with imparted modulation, in accordance with someimplementations of the present disclosure. Method 800 can be used incombination with method 700 (e.g., as part of block 760 of method 700).Method 800 can be performed by components of the hybrid lidar thatperform some of the operations of method 700. In some implementations,some of the components performing method 800 can be different fromcomponents that perform method 700. Method 800 will be illustrated belowusing an example implementation that deploys frequency modulation, butit should be understood that substantially similar operations can beperformed in implementations where phase or amplitude modulation isused. Method 800 can include determining, at block 810, that a currentfrequency, f_(LO)(t), of a local oscillator (LO) copy of the second beamis associated (e.g., f_(LO)(t)=f_(k)) with a first interval (e.g., k-thinterval or plateau) of the modulation imparted to the second beam. Atblock 820, method 800 can continue with determining that a frequency ofthe second RX signal (e.g., the frequency of the continuous beam returnf_(RX)(t)) is associated with a second interval (e.g., j-th interval orplateau) of the modulation imparted to the second beam (e.g., thatf_(RX)(t) is close to f_(j), with the difference representing theDoppler shift). At block 830, method 800 can continue with determining aminimum distance to the object L_(MIN) and a maximum distance to theobject L_(MAX) using a time delay between the first interval and thesecond interval (e.g., the number of intervening modulationintervals/plateaus between the j-th interval/plateau and the k-thinterval/plateau, as described in conjunction with FIGS. 5-6 . In someimplementations, blocks 810-830 may be performed (e.g., for improvedresolution and/or reliability of detections) using any combination ofphase modulation, frequency modulation, and amplitude modulation basedon time delays between corresponding modulation intervals of the RXsignals and LO copy of the continuous part of the TX beam.

Some portions of the detailed descriptions above are presented in termsof algorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise, as apparent from the followingdiscussion, it is appreciated that throughout the description,discussions utilizing terms such as “identifying,” “determining,”“storing,” “adjusting,” “causing,” “returning,” “comparing,” “creating,”“stopping,” “loading,” “copying,” “throwing,” “replacing,” “performing,”or the like, refer to the action and processes of a computer system, orsimilar electronic computing device, that manipulates and transformsdata represented as physical (electronic) quantities within the computersystem's registers and memories into other data similarly represented asphysical quantities within the computer system memories or registers orother such information storage, transmission or display devices.

Examples of the present disclosure also relate to an apparatus forperforming the methods described herein. This apparatus can be speciallyconstructed for the required purposes, or it can be a general purposecomputer system selectively programmed by a computer program stored inthe computer system. Such a computer program can be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding optical disks, CD-ROMs, and magnetic-optical disks, read-onlymemories (ROMs), random access memories (RAMs), EPROMs, EEPROMs,magnetic disk storage media, optical storage media, flash memorydevices, other type of machine-accessible storage media, or any type ofmedia suitable for storing electronic instructions, each coupled to acomputer system bus.

The methods and displays presented herein are not inherently related toany particular computer or other apparatus. Various general purposesystems can be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear as set forth in thedescription below. In addition, the scope of the present disclosure isnot limited to any particular programming language. It will beappreciated that a variety of programming languages can be used toimplement the teachings of the present disclosure.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other implementation exampleswill be apparent to those of skill in the art upon reading andunderstanding the above description. Although the present disclosuredescribes specific examples, it will be recognized that the systems andmethods of the present disclosure are not limited to the examplesdescribed herein, but can be practiced with modifications within thescope of the appended claims. Accordingly, the specification anddrawings are to be regarded in an illustrative sense rather than arestrictive sense. The scope of the present disclosure should,therefore, be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

What is claimed is:
 1. A system comprising: a first light sourceconfigured to produce a first beam comprising one or more pulses; asecond light source configured to produce a second beam, wherein thesecond beam is a continuous beam; a modulator configured to impart amodulation to the second beam; an optical interface subsystem configuredto: transmit the first beam and the second beam to an outsideenvironment, and receive, from the outside environment, a plurality ofreceived (RX) signals caused by at least one of the first beam or thesecond beam; and one or more circuits configured to determine that afirst RX signal of the plurality of RX signals and a second RX signal ofthe plurality of RX signals are reflected by a same object, wherein thefirst RX signal is representative of a distance to the object and thesecond RX signal is representative of (i) a velocity of the object, and(ii) an interval of possible distances to the object, the interval ofpossible distances being identified based on the modulation of thesecond RX signal.
 2. The system of claim 1, wherein the modulationimparted to the second beam comprises at least one of (i) a temporalsequence of frequency shifts, (ii) a temporal sequence of phase shifts,or (iii) a temporal sequence of amplitude changes.
 3. The system ofclaim 2, wherein the modulation imparted to the second beam comprises atleast four different frequency shifts or four different phase shifts. 4.The system of claim 1, wherein the modulator is one of an acousto-opticmodulator or an electro-optic modulator.
 5. The system of claim 4,wherein the modulation is imparted by an acoustic wave induced in theacousto-optic modulator.
 6. The system of claim 1, wherein the firstbeam has a first spectral content and the second beam has a secondspectral content that is different from the first spectral content. 7.The system of claim 6, wherein the first beam has a first centralwavelength and the second beam has a second central wavelength that isdifferent from the first central wavelength of the first beam.
 8. Thesystem of claim 1, wherein the optical interface subsystem is configuredto transmit the first beam and the second beam along the same opticalpath.
 9. The system of claim 1, further comprising a coherentphotodetector configured to generate an electronic signal representativeof a difference between the modulation of the second RX signal and themodulation of the second beam.
 10. The system of claim 9, wherein afirst optical input into the coherent photodetector is the second RXsignal and a second optical input into the coherent photodetector is alocal oscillator copy of the second beam.
 11. The system of claim 1,wherein the one or more circuits are configured to: determine that acurrent frequency or a phase shift of a local oscillator (LO) copy ofthe second beam is associated with a first interval of the modulationimparted to the second beam; determine that a frequency or a phase shiftof the second RX signal is associated with a second interval of themodulation imparted to the second beam; and determine a minimum distanceto the object and a maximum distance to the object using a time delaybetween the first interval and the second interval.
 12. The system ofclaim 1, further comprising a processing device configured to: associatethe distance to the object, determined from the first RX signal, withthe velocity of the object, determined from the second RX signal, basedon the distance to the object being within the interval of possibledistances to the object, determined from the second RX signal.
 13. Asensing system of an autonomous vehicle (AV), the sensing systemcomprising: an optical system configured to: produce a first beamcomprising one or more pulses, the first beam centered at a firstfrequency; produce a second beam, wherein the second beam is acontinuous beam centered at a second frequency different from the firstfrequency; impart an angle modulation to the second beam; transmit thefirst beam and the second beam to an environment of the AV, and receive,from the environment of the AV, a plurality of received (RX) signalscaused by at least one of the first beam or the second beam; and asignal processing system configured to: determine that a first RX signalof the plurality of RX signals and a second RX signal of the pluralityof RX signals are reflected by a same object in the environment of theAV, wherein the first RX signal is representative of a distance to theobject and the second RX signal is representative of (i) a velocity ofthe object, and (ii) an interval of possible distances to the object,the interval of possible distances being identified based on the anglemodulation of the second RX signal; and associate the distance to theobject, determined from the first RX signal, with the velocity of theobject, determined from the second RX signal.
 14. The sensing system ofthe AV of claim 13, wherein the angle modulation imparted to the secondbeam comprises at least one of (i) a temporal sequence of frequencyshifts, or (ii) a temporal sequence of phase shifts.
 15. The sensingsystem of the AV of claim 13, wherein the signal processing systemfurther comprises a coherent photodetector to generate an electronicsignal representative of a difference between the angle modulation ofthe second RX signal and the angle modulation of the second beam.
 16. Amethod comprising: producing a first beam comprising one or more pulses;producing a second beam, wherein the second beam is a continuous beam;imparting a modulation to the second beam; transmitting the first beamand the second beam to an outside environment; receiving, from theoutside environment, a plurality of received (RX) signals caused by atleast one of the first beam or the second beam; and determining that afirst RX signal of the plurality of RX signals and a second RX signal ofthe plurality of RX signals are reflected by a same object, wherein thefirst RX signal is representative of a distance to the object and thesecond RX signal is representative of (i) a velocity of the object, and(ii) an interval of possible distances to the object, the interval ofpossible distances being identified based on the modulation of thesecond RX signal.
 17. The method of claim 16, wherein the modulationimparted to the second beam comprises at least one of (i) a temporalsequence of frequency shifts, or (ii) a temporal sequence of phaseshifts.
 18. The method of claim 16, wherein the first beam and thesecond beam are transmitted along a same optical path.
 19. The method ofclaim 16, wherein determining that the first RX signal and the second RXsignal are reflected by the same object comprises: determining that acurrent frequency or a phase shift of a local oscillator (LO) copy ofthe second beam is associated with a first interval of the modulationimparted to the second beam; determining that a frequency or a phaseshift of the second RX signal is associated with a second interval ofthe modulation imparted to the second beam; and determining a minimumdistance to the object and a maximum distance to the object using a timedelay between the first interval and the second interval.
 20. The methodof claim 16, further comprising: associating the distance to the object,determined from the first RX signal, with the velocity of the object,determined from the second RX signal, based on the distance to theobject being within the interval of possible distances to the object,determined from the second RX signal.